stacked on #625

Live Query Scheduler Overview

Overview

This change introduces a scoped, dependency-aware scheduler that guarantees every live-query builder runs at most once per transaction, even when multiple source collections or derived queries fire in the same mutate call. The scheduler groups work by transaction id, dedupes entries by the CollectionConfigBuilder instance, tracks dependencies between builders, and flushes synchronously when Transaction.mutate exits. The net effect: optimistic updates from a single transaction coalesce into a single graph run for each live query, so downstream frameworks see one change batch per transaction.

How scheduling works

• Each CollectionSubscriber immediately forwards changes to the D2 input and calls CollectionConfigBuilder.scheduleGraphRun.
• scheduleGraphRun records which upstream builders the current builder depends on (based on the collections it subscribed to) and hands a job to the shared scheduler with:
  • contextId: the transaction id, grouping all work triggered by that transaction.
  • jobId: the builder instance; repeated schedules for the same builder merge into a single entry.
  • dependencies: the set of upstream builders that must finish before this builder can run.
• The scheduler maintains, per transaction, a queue, entry map, dependency map, and “completed” set. When flushing, it only executes a job once all dependencies are marked completed, deferring the job otherwise. It loops until no work remains and throws if it detects an impossible cycle.

When the graph actually runs

• Inside Transaction.mutate, we call registerTransaction before the user’s callback and unregisterTransaction in a finally block afterward.
• registerTransaction clears any stale entries from previous failed scopes.
• unregisterTransaction calls scheduler.flush(tx.id). flush now loops while a context map exists, so if a job enqueues additional work during its own execution (e.g., the join scheduling itself after its parents run), the scheduler immediately picks up the new entry before leaving the transaction scope.
• If there is no active transaction, scheduleGraphRun runs maybeRunGraph immediately (backward compatible behaviour).

Nested live queries (the “diamond” cases)

• Builders register themselves when a live-query collection is created. Whenever a builder subscribes to another live query, it records that dependency.
• During a transaction, updates enqueue jobs for the parent builders. When those builders finish, they mark themselves as completed, which in turn allows child jobs (such as joins) to run exactly once at the end of the flush.
• A hybrid variant (join between liveQueryA and raw collectionB) behaves the same way: even if collectionB fires first, the join job is deferred until liveQueryA has finished.
• Because dependencies are tracked explicitly, there are no deadlocks. If the scheduler ever loops without making progress, it throws a clear “dependency cycle” error.

Order-by loaders and batching

• The maybeRunGraph loop keeps calling graph.run() while pendingWork() is true. If an order-by loader requests more rows, it pushes those changes into the same D2 input, triggering pendingWork() again.
• During this loop CollectionConfigBuilder.isGraphRunning is true, so any nested call to scheduleGraphRun is ignored; the loader’s extra rows are consumed by the ongoing loop.
• As a result, even hungry top-K queries emit a single batch per synchronous transaction run. Multiple batches only occur when a loader deliberately yields results asynchronously (for example, the incremental sync features we’re adding)—in that case the UI should expect staged updates.

Tests

scheduler.test.ts now covers:

1. Basic single run per transaction (two collections mutated inside one mutate).
2. Nested transactions.
3. Rollback cleanup.
4. Multiple subscribers receiving exactly one batch.
5. Loader deduping.
6. Diamond dependency (live query joining two parents) – verifies parents run first and the join runs once (tracked via getRunCount).
7. Hybrid diamond (join of a live query with a base collection) – same single-run guarantee, even when the raw collection fires first.

Each of the diamond tests mutates once and then performs another transaction with updates, demonstrating that we still emit exactly one additional batch—no duplicates, no missed runs—and that getRunCount() only increments once per transaction.

Summary

• Scheduler coalesces work by transaction + builder.
• Transaction.mutate clears before and flushes after every scope, so everything runs synchronously in the optimistic phase.
• flush loops until no jobs remain, ensuring child live queries (joins) run exactly once per transaction after their parents.
• Comprehensive tests cover simple, nested, rollback, multi-subscriber, loader, diamond, and hybrid patterns; asynchronous loaders will still produce multiple batches by design.